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Abstract 

Background: Because of a physician shortage in many low-income countries, the use of nonphysicians to classify 
perinatal mortality (stillbirth and early neonatal death) using verbal autopsy could be useful. 

Objective: To determine the extent to which underlying perinatal causes of deaths assigned by nonphysicians in 
Guatemala, Pakistan, Zambia, and the Democratic Republic of the Congo using a verbal autopsy method are 
concordant with underlying perinatal cause of death assigned by physician panels. 

Methods: Using a train-the-trainer model, 13 physicians and 40 nonphysicians were trained to determine cause of 
death using a standardized verbal autopsy training program. Subsequently, panels of two physicians and individual 
nonphysicians from this trained cohort independently reviewed verbal autopsy data from a sample of 118 early 
neonatal deaths and 134 stillbirths. With the cause of death assigned by the physician panel as the reference 
standard, sensitivity, specificity, positive and negative predictive values, and cause-specific mortality fractions were 
calculated to assess nonphysicians' coding responses. Robustness criteria to assess how well nonphysicians 
performed were used. 

Results: Causes of early neonatal death and stillbirth assigned by nonphysicians were concordant with physician- 
assigned causes 47% and 57% of the time, respectively. Tetanus filled robustness criteria for early neonatal death, 
and cord prolapse filled robustness criteria for stillbirth. 

Conclusions: There are significant differences in underlying cause of death as determined by physicians and 
nonphysicians even when they receive similar training in cause of death determination. Currently, it does not 
appear that nonphysicians can be used reliably to assign underlying cause of perinatal death using verbal autopsy. 



Background medical certification of COD [3]. By contrast, many 

Understanding population-based causes of perinatal low- and middle- income countries, which have the 
death (stillbirth [SB] and early neonatal deaths [END], i. highest burden of poverty and disease, continue to lack 
e., newborn deaths in the first seven days of life) is routine, representative, and high-quality information on 
essential when developing an effective perinatal health COD and population-based cause-specific mortality frac- 
policy [1]. Because there will always be competing tions (CSMF) [4]. Fewer than 3% of all perinatal deaths 
demands for health care resources, a robust system con- in low- and middle-income countries have medical certi- 
structed to identify and assign a medically-determined fication of COD [5]. In part, this may be because more 
cause of death (COD) for each perinatal death is highly than half of all births and perinatal deaths occur in the 
desirable [2]. In many high-income countries, there is a home and are frequently unrecorded in vital registration 
complete record of each death, and 90% of these have or health systems [6]. 

Increasing numbers of low- and middle- income coun- 
tries are using verbal autopsy (VA) methods as an epide- 
miologic tool to inform mortality surveillance systems 
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relies on information obtained from an interview with 
the primary caregiver (usually the mother) of the 
deceased. During this process, the symptoms, signs, and 
behaviors during the illness of the deceased, or of the 
mother in the case of fetal death, are recorded [8]. This 
information is summarized and reviewed and the most 
probable COD assigned. VA is proving to be a cost- 
effective, practical, and sustainable alternative to a thor- 
ough medical diagnostic evaluation where vital registra- 
tion systems are weak [9]. 

A variety of methods exist for interpreting VA inter- 
views to arrive at a COD. The most commonly used 
method has two or three trained physician coders review 
the data and independently assign a COD [10]. Any dis- 
crepancies between the COD assigned by each physician 
member of the panel are resolved by discussion and 
review of the VA data, and a final consensus COD is 
agreed upon by the physician panel. Alternatively, COD 
can be assigned by the use of predetermined criteria/ 
algorithms, computer simulations, or probabilistic 
approaches, all of which do not require the presence of 
a physician [11-15]. 

There is a widespread physician shortage in many 
low-income countries and significant costs incurred in 
recruiting, training, and utilizing physicians. Reports 
suggest that nonphysician providers can conduct speci- 
fied clinical tasks with adequate training [16-18]. We 
previously reported that when taught a standardized VA 
package in a classroom setting, nurses and midwives 
achieve a level of cognitive and applied knowledge com- 
parable to physicians in determining perinatal COD 
[19]. Thus, we sought to investigate whether, following 
this training, nonphysicians can determine causes of SB 
and END in rural communities as reliably as physicians. 

Methods 

Setting, subjects, and study design 

This prospective observational study was nested within 
the FIRST BREATH Trial conducted by the Eunice Ken- 
nedy Shriver National Institute of Child Health and 
Human Development Global Network for Women's and 
Children's Health Research [20]. The FIRST BREATH 
Trial was a cluster randomized, controlled trial that 
investigated the effects of implementing a package of 
newborn care practices and newborn resuscitation in 
community settings. 

This VA study included 38 communities from Guate- 
mala (Chimaltenango province), the Democratic Repub- 
lic of the Congo (DRC) (Equateur province), Zambia 
(Kafue district), and Pakistan (Thatta district). Each 
community comprised a cluster of villages with approxi- 
mately 300 deliveries per year. Data describing births 
were collected by birth attendants and reviewed by 
trained nurse-midwives (with three to four years of 



health training) or community health workers (high 
school graduates with 18 months of health training) 
designated as community coordinators. Within one 
week of an END or SB, birth attendants notified com- 
munity coordinators who then visited the family, deter- 
mined eligibility for the study, and requested consent 
from eligible mothers. Perinatal deaths were excluded if 
they occurred in a hospital, if a birth attendant was not 
present at delivery, if the mother was unavailable for 
any reason (including peripartum death), or if the 
mother could not be enrolled within seven days of 
death. A seven-day enrollment window was chosen to 
reduce the variability in the quality of reporting intro- 
duced by recall bias [21-23]. Because the conventional 
perinatal VA respondents are mothers, we included only 
those subjects whose mothers were available for inter- 
view. Informed consent was obtained from mothers in a 
private and confidential setting. The consent form was 
read to all mothers who then provided their signatures 
or, if they were illiterate, thumbprints. 

Training and VA methodology 

Neither community coordinators nor physicians had 
prior experience before the study with the use of VA to 
determine COD. All community coordinators and physi- 
cians participating in this study received standardized 
training in VA methods over three days, via a train-the- 
trainer method [19]. Community coordinators were 
trained to interview mothers using the VA question- 
naire. To assign underlying COD, both community 
coordinators and physicians were trained in the classifi- 
cation, rules, and guidelines of the 10th revision of the 
International Classification of Diseases (ICD-10). Under- 
lying COD was defined as the single most important 
disease or condition that initiated the train of morbid 
events leading directly to fetal or neonatal death. 

Uniform data describing the circumstances surround- 
ing a perinatal death were collected from each mother 
using a standardized VA questionnaire developed speci- 
fically for this study from a validated VA tool [24] . The 
questionnaire was administered by the community coor- 
dinators who then sent these data separately to two 
local physicians. Additionally, the community coordina- 
tors and physicians were provided with demographic 
and other descriptive data collected as part of the FIRST 
BREATH Trial. Each community coordinator and physi- 
cian independently first determined whether the death 
occurred prior to birth and was therefore classified as a 
SB, or after a live birth and classified as an END. Then 
they assigned one underlying COD. After the COD was 
assigned and entered independently, any discrepancy in 
assignment of COD between physicians was discussed 
and consensus underlying COD was assigned. The 
underlying COD assigned by the community 
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coordinator was then compared to the consensus under- 
lying COD assigned by the physician panel. 

Data collection and analysis 

Data were entered and transmitted electronically to the 
data coordinating center (Research Triangle Institute, 
Research Triangle Park, NC, USA) where data edits, 
including inter- and intraform consistency checks, were 
performed. The study was reviewed and approved by 
the institutional ethics review committees of the 
Research Triangle Institute, the University of North Car- 
olina at Chapel Hill, and in-country Institutional Review 
Boards. 

Data were analyzed using SAS (SAS/STAT Software 
version 9.2). Physician perinatal COD responses were 
viewed as the reference standard for calculations of sen- 
sitivity, specificity, positive and negative predictive 
values, and CSMF, which were calculated using conven- 
tional two-by-two table analysis. The Delta method was 
used to calculate confidence intervals for the CSMFs 
[25]. We defined the CSMF as the number of perinatal 
deaths (END or SB) due to a specific cause divided by 
the total number of deaths. 

Before the start of the study, our a priori hypothesis 
was that the COD assigned by community coordinators 
would be concordant with the COD assigned by the 
physician panel in greater than 70% of perinatal deaths, 
and we powered our study accordingly. We also 



9461 Births 



assessed the degree of robustness of community coordi- 
nator responses. We defined robustness using criteria 
previously described, utilized, and published by Setel et 
al [26]. To be considered "robust" a condition must 
meet the following criteria: 1) sensitivity > 50%, 2) speci- 
ficity > (1-CSMF of the physician consensus), and 3) 
relative difference between the CSMF for the commu- 
nity coordinator and the CSMF for the physician con- 
sensus within 20%. The relative difference was 
calculated as follows: absolute value ((CSMF of the phy- 
sician consensus - CSMF of the community coordina- 
tor)/CSMF of the physician consensus x 100%). 
Additionally, we calculated the level of agreement 
between the physician consensus and community coor- 
dinators, using Cohen's kappa statistic. Levels of agree- 
ment based on ranges of kappa values were defined as 
follows: 0.81-0.99, almost perfect agreement; 0.61-0.80, 
substantial agreement; 0.41-0.60 moderate agreement; 
and less than 0.4, slight to fair agreement [24]. 

Results 

The study period was from May 2007 to June 2008, dur- 
ing which 9,461 infants were born in the designated 
communities. Among these, birth attendants identified 
518 SB and END (Figure 1). The SB, END, and perinatal 
mortality rates were 30/1000 births, 25/1000 live births, 
and 55/1000 births, respectively. Of the 518 deaths, 81 
were ineligible for the study because the delivery 



518 Perinatal Deaths 

• 229 Early neonatal deaths 

• 289 Stillbirths 



f 

437 Elig 


\ 

ible Deaths 






252 Enrolled 

v / 



81 Ineligible 

Birth attendant absent at delivery 2 
Infant delivered in a hospital 79 



185 Not Enrolled 

Mother did not provide consent 40 
Mother unavailable for interview within 
seven days after the death 145 



134 Stillbirths 



118 Early Neonatal 
Deaths 



Figure 1 The verbal autopsy study population 
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occurred in a hospital (79) or the birth attendant was 
absent at the time of delivery (2). Among eligible deaths, 
185 were not enrolled because the mother was not avail- 
able for interview within seven days after the death 
(145) or did not provide consent (40). This study reports 
on 252 perinatal deaths (134 SBs and 118 ENDs), based 
on determinations by the physicians regarding the tim- 
ing of perinatal deaths. 

Concordance of stillbirth and early neonatal death 
between physicians and nonphysicians 

Ninety-three percent of perinatal deaths determined by 
physicians to be SBs were classified as SBs by commu- 
nity coordinators; the remainder were classified as 
ENDs. Ninety-five percent of perinatal deaths deter- 
mined by physicians to be ENDs were classified by com- 
munity coordinators as ENDs; the remainder were 
classified as SBs. Concordance between physicians and 
community coordinators in the determination of timing 
of perinatal deaths did not vary between the two classes 
of community coordinators (nurse-midwives and com- 
munity health workers). 

Early neonatal death 

Table 1 compares underlying causes of END assigned by 
physician panels and community coordinators. Overall, 
causes of END assigned by community coordinators 
were concordant with causes of END assigned by physi- 
cian panels 47% of the time. Table 2 describes the sensi- 
tivity, specificity, positive and negative predictive values, 
and CSMF of specific underlying causes of END 
assigned by community coordinators. Kappa values are 
additionally included. Sensitivity and specificity were 
high for preterm/low birth weight and tetanus. By con- 
trast, sensitivity was low for infections and asphyxia, 
although specificity for both of these was 0.90 or above. 



The positive predictive values for infections and tetanus 
were 0.83 and 0.67, respectively, and the negative pre- 
dictive values for preterm/low birth weight, trauma, and 
tetanus were 0.93 or above. The relative difference 
between CSMF assigned by physician panels and com- 
munity coordinators was 20% for tetanus and > 20% for 
all other diagnostic categories. Only the diagnosis of 
tetanus fulfilled criteria for robustness. When the level 
of agreement among the different diagnostic categories 
was considered using Cohen's kappa statistic, there was 
substantial agreement for the diagnosis of tetanus (0.71); 
all other categories showed slight or only moderate 
agreement. 

Stillbirth 

Table 3 summarizes the comparison of community 
coordinator and physician underlying COD for SB. 
Overall, causes of SB assigned by community coordina- 
tors were concordant with causes of SB assigned by phy- 
sician panels 57% of the time. Table 4 presents the 
sensitivity, specificity, positive and negative predictive 
values, and CSMF of underlying SB COD assigned by 
community coordinators. Kappa values are additionally 
included. Sensitivity for antepartum hemorrhage and 
maternal infection was 0.62 and 0.64, respectively, and 
for all other known COD categories was 0.57 or less. 
Specificity was generally high (0.94 or higher) except for 
prematurity, for which the specificity was 0.86. The 
positive predictive values of antepartum hemorrhage, 
maternal infection, maternal accident, and prolonged 
labor were 0.80 or higher. Maternal infections had a 
negative predictive value of 0.81; all the other SB COD 
categories had values of 0.94 or above. The relative dif- 
ference between CSMF assigned by physician panels and 
community coordinators was 13% for cord prolapse; all 
other SB diagnostic categories were greater than 20%. 



Table 1 Comparison of physician consensus (PC) and community coordinator (CC) for underlying neonatal cause of 
death (COD) 



CC for underlying cause of death PC for underlying cause of death CC 

total 





Preterm/ 
Low birth weight 




Asphyxia 


Fetal Trauma 


Tetanus 


Unknown/ 
no cause 


Other 1 




Preterm/low birth weight 


14 


11 


3 


0 


1 


0 


2 


31 


Infection 


1 


19 


3 


0 


0 


0 


0 


23 


Asphyxia 


4 


5 


13 


0 


0 


0 


0 


22 


Fetal trauma 


0 


4 


3 


0 


0 


0 


0 


7 


Tetanus 


0 


2 


0 


0 


4 


0 


0 


6 


Unknown/no cause 


0 


6 


3 


0 


0 


2 


1 


12 


Other 1 


1 


5 


6 


0 


0 


2 


3 


17 


PC total 


20 


52 


31 


0 


5 


4 


6 


118 



1 0ther includes congenital malformation, birth trauma, and neonatal accident, 
Percent concordant = (55/1 18) x 100 = 46.6%. (This includes COD coded as Other.) 



Table 2 Comparison of neonatal underlying cause 
(CC) for select causes of early neonatal death (n = 


reported by physician consensus (PC) to underlying cause of death reported by community coordinator 
118) 


Underlying COD, as determined by CC 


Underlying COD, as determined by PC (reference 
standard) 




Measures 2 












Identified 
as LUU 




Not identified 
as luu 


SE SP PPV NPV CSMFpc 
(95% CI) 


1 -CSMFpc 


CSMFcc 
(95% CI) 


1 -CSMFcc 


RD 

(95% CI) 


Kappa 
(95 CI) 


Preterm/LBW 


20 




98 


0.70 0.83 0.45 0.93 0.17 

(u.uy, U.ZDj 


0.83 


0.26 
(U. I o, U.o/J 


0.74 


0.55 

IC\ f\Q 1 C\A\ 

(,U.uo, I .U4) 


0.43 

(U.Z4, U.oz] 


COD 


14 (70.0) 




17 (17.3) 














Not COD 


6 (30.0) 




0 1 fQ~i ~7\ 














Infection 


52 




66 


0.37 0.94 0.83 0.65 0.44 

(0.29, o.by) 


0.56 


0.19 
(0. I I , O.zo) 


0.81 


0.56 
(0.40, 0./0J 


0.32 
(0.1 /, 0.48) 


COD 


19 (36.5) 




4 (6.1) 














Not COD 


33 (63.5) 


















Asphyxia 


31 




87 


0.42 0.90 0.59 0.81 0.26 

{{). I O, U.i/j 


0.74 


0.19 
{{). I U, U.z/J 


0.81 


0.29 
(O.Oo, U.54) 


0.35 

(U. I 3, U.34] 


COD 


13 (41.9) 




9 (10.3) 














Not COD 


1 q /roil 
1 O [DO. 1 J 




7o /on ~7\ 














Fetal Trauma 


0 




118 


- 0.94 0.00 1.00 


- 


0.06 
(U.U 1 , U. 1 1 J 


0.94 


- 




COD 


0 (0.0) 




7 (5.9) 














Not COD 


0 (0.0) 




111 !CiA 1 \ 














Tetanus 


5 




113 


0.80 0.98 0.67 0.99 0.04 

ir\ r\r\ r\ r\Q\ 

(0.U0, U.Oo) 


0.96 


0.05 

\u.{) i , u.uyj 


0.95 


0.20 
(,U.(JU,z.UUj 


0.71 
U.4 1 , I.UUJ 


COD 


4 (80.0) 




2 (1.8) 














Not COD 


1 (200) 




1 1 1 (93 2) 














Unknown/ 

n ^ r^i icq 


4 




114 


0.50 0.91 0.17 0.98 0.03 

(0 00 0 07) 
\U.UU, U.U/ ) 


0.97 


0.10 

^U.U'H-, U. 1 \J) 


0.90 


2.00 

(Oil 11 01 
\U.Z 1 , II .UJ 


0.21 


COD 


2 (50.0) 




10 (8.8) 














Not COD 


2 (50.0) 




104 (91.2) 














Other 1 


6 




112 


0.50 0.88 0.18 0.97 0.05 

(0.01, 0.09) 


0.95 


0.14 
(0.07, 0.22) 


0.86 


1.83 
(0.38,7.33) 


0.20 
(-0.04, 0.44) 


COD 


3 (50.0) 




14 (12.5) 














Not COD 


3 (50.0) 




98 (87.5) 















1 Other includes congenital malformation, birth trauma, and neonatal accident. 

2 Definitions of measures are provided in the methods section. Measures are as follows: sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), cause-specific mortality fraction 
(CSMF) of physician consensus (PC) and community coordinator (CC), and absolute relative difference of the CSMF (RD). RD was calculated using the actual numeric values rather than the rounded CSMFs reported in 
the table. 
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Table 3 Comparison of physician consensus (PC) and community coordinator (CC) for underlying maternal cause of 
stillbirth 



CC for underlying PC underlying maternal cause of death CC 

maternal cause of death total 





Antepartum 
hemorrhage 




Prematurity 


Accident 


Prolonged 
labor 


Cord prolapse/ 
complication 


Malpre- 
sentation 


Unknown/ 
no cause 


Other 1 




Antepartum hemorrhage 


8 


1 


0 


0 


1 


0 


0 


0 


0 


10 


Maternal infection/sepsis 


1 


32 


0 


0 


0 


1 


0 


2 


1 


37 


Prematurity 


3 


9 


4 


1 


0 


0 


0 


2 


2 


21 


Maternal accident 


0 


1 


0 


4 


0 


0 


0 


0 


0 


5 


Prolonged labor 


0 


0 


0 


0 


8 


0 


2 


0 


0 


10 


Cord prolapse/ 
complication 


1 


2 


0 


0 


2 


4 


0 


0 


0 


9 


Malpresentation 


0 


1 


0 


0 


0 


0 


2 


0 


2 


5 


Unknown/no cause 


0 


3 


5 


2 


1 


1 


0 


11 


4 


27 


Other 1 


0 


1 


0 


0 


3 


2 


0 


1 


3 


10 


PC total 


13 


50 


9 


7 


15 


8 


4 


16 


12 


134 



1 Other includes multiple delivery, hypertension/eclampsia, and post-term delivery 
Percent concordant = (76/134) x 100 = 56.7%. (This includes COD coded as Other.) 



Cord prolapse fulfilled the robustness criteria. It is 
worth pointing out that where relative difference values 
are small, it may be because there are very small 
CSMFs. When the level of agreement among the differ- 
ent diagnostic categories was considered using Cohen's 
kappa statistic, no category demonstrated almost perfect 
agreement. 

Discussion 

There are three main findings from this study. The first 
is that given identical data from the VA questionnaires, 
community coordinators and physicians draw the same 
conclusions about the timing of perinatal death (SB and 
END) 95% of the time. Second, causes of SB and END 
assigned by community coordinators were concordant 
with causes of SB and END assigned by physician panels 
57% and 47% of the time, respectively. Third, only one 
cause of END assigned by community coordinators 
(tetanus) met robustness criteria. Similarly, when robust- 
ness criteria were applied to SB diagnostic categories to 
assess the performance of community coordinators, only 
cord prolapse met criteria. 

Task-shifting of physician-domain responsibilities is an 
increasingly important concept that is gaining support in 
the literature [16,17]. Numerous authors have assessed 
the utilization and impact of nonphysician providers 
after being taught a structured curriculum [18,19]. 
These authors report that nonphysicians, specifically 
nurse-midwives, can perform comparably to physicians 
when taught a structured teaching program with ade- 
quate supervision. To our knowledge, only one study 
has compared nonphysicians to physicians in determin- 
ing perinatal COD in the field using verbal autopsy 



methods, and none has compared nonphysicians from a 
variety of countries with a range of backgrounds such as 
nurse-midwives in Zambia and DRC and community 
health workers in Pakistan and Guatemala [27]. Our 
group previously examined how well community coordi- 
nators and physicians performed when taught a struc- 
tured VA program in a classroom setting [19]. In both 
cognitive and applied knowledge, community coordina- 
tors' pretest results were lower than physicians; however, 
these results improved significantly post-test, with 
nurse-midwives showing comparable results to physi- 
cians. In light of these data we undertook the present 
study. Our study showed that despite the ability to 
improve cognitive and applied knowledge in the class- 
room setting, this knowledge did not result in nonphysi- 
cians reaching similar conclusions about COD in actual 
practice. 

Chowdhury et al. reported on the use of medical assis- 
tants (with three years of institutional training) in a sin- 
gle site in Matlab, Bangladesh, to interpret neonatal VA 
data and assign COD [27]. When specific diagnostic 
categories assigned by medical assistants were compared 
to physician panels, birth asphyxia showed good reliabil- 
ity with kappa values of 0.77, while prematurity, respira- 
tory distress syndrome, pneumonia, and sepsis/ 
meningitis showed moderate agreement, with kappa 
values between 0.51 and 0.59. The authors concluded 
that medical assistants are generally knowledgeable 
about the disease profile of a geographic area, can gen- 
erally use their clinical judgment and knowledge to 
determine COD for all ICD-10 classes of neonatal 
death, and may be considered an alternative for deter- 
mining neonatal COD in rural areas where physicians 



Table 4 Comparison of maternal underlying cause reported by physician consensus (PC) to underlying cause reported by community coordinator (CC) for 
select causes of stillbirth (n = 134) 



Underlying COD, as determined by CC 


Underlying COD, as determined by PC (reference 
standard) 










Measures 


2 










Identified 

as LUU 


Not identified 
as luu 


SE 


SP PPV 


NPV 


CSMFpc 
(95% CI) 


1 -CSMFpc 


CSMFcc 
(95% CI) 


1 -CSMFcc 


RD 

(95% CI) 


Kappa 
(95% CI) 


Antepartum hemorrhage 


13 


121 


0.62 


0.98 0.80 


0.96 


0.10 

\<J.\JH, U. 1 D) 


0.90 


0.07 

^U.Uj, u. I ZJ 


0.93 


0.23 
(o nn n ^7~\ 

\U.UU,U.J / j 


0.67 
(0 a a n Q~\ 


COD 


8 (61.5) 


2 (1.7) 




















Not COD 


D [30.D) 


1 1 Q /OQ ^ 

i i y (.yo.jj 




















Maternal infection 


50 


84 


0.64 


0.94 0.86 


0.81 


0.37 

\U.ZD, U.DUJ 


0.63 


0.28 

(r\ 1 7 n 

\U. \ 1 , <J.DO) 


0.72 


0.26 

[U.O / ,U.HZ) 


0.62 

^U/H-/ ,U./ 


COD 


32 (64.0) 


5 (6.0) 




















Not COD 


1 o po.Uj 


7Q /'Q/1 rrt 




















Prematurity 


9 


125 


0.44 


0.86 0.19 


0.96 


0.07 
(oc\i nil 1 ) 

^U.UZ, U. 1 1 j 


0.93 


0.16 
(ono n"?31 

l,U.Uo, U.ZoJ 


0.84 


1.33 
/"n 1A A ACS\ 


0.19 
\ u.uz,u.^ 


COD 


4 (,44 .4 J 


1 7 (1 3.6) 




















Not COD 


D [DD.O) 


I Uo \oO.Hj 




















Maternal accident 


7 


127 


0.57 


0.99 0.80 


0.98 


0.05 

(r\ n 1 n noi 
\u.\j \ , u.uyj 


0.95 


0.04 
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1 Other includes multiple delivery, hypertension/eclampsia, post-term delivery, and other causes. 

2 Definitions of measures are provided in the methods section. Measures are as follows: sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), cause-specific mortality fraction 
(CSMF) of physician consensus (PC) and community coordinator (CC), and absolute relative difference of CSMF (RD). RD was calculated using the actual numeric values rather than the rounded CSMFs reported in 
the table. 
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are scarce. A number of reasons may explain the differ- 
ences observed between our study and those of 
Chowdhury et al. First, in our study causes of both SB 
and END were assigned, in contrast to neonatal out- 
comes only from Matlab. Also, community coordinators 
had different educational backgrounds. Nurses and mid- 
wives had three to four years post-high school health 
training, whereas community health workers had only 
18 months post-high school health training, in contrast 
to medical assistants from Matlab who had three years 
training. The community coordinators in our study had 
no a priori experience with VA and assigning COD, 
unlike the Matlab cohort, and our study was a multisite 
study compared to the single site of Matlab, where the 
medical assistants over the years have been closely 
involved in verbal autopsy work in the demographic sur- 
veillance program. 

The concept of underlying COD, the single most 
important disease or condition that initiated the chain 
of events leading directly to fetal or neonatal death, is 
complex. It requires a deep appreciation of pathophy- 
siology and, especially in the case of perinatal death, 
consideration of both the mother and the fetus. To 
effectively utilize this concept, the coder has to "con- 
struct a story" of what happened. The key initiating fac- 
tor, without which the death most likely might not have 
occurred, is the cornerstone of the "story." It is possible 
that community coordinator responses might reflect 
other categories of COD, such as the final and contri- 
buting COD as described in the ICD-10. Further 
research is needed to determine whether concordance of 
community coordinator-assigned COD when utilizing 
multiple or other categories of COD might yield higher 
concordance with physician panels. 

Comparing only underlying COD may be a potential 
limitation in this study. It appears increasingly evident 
that individual perinatal deaths in low-income countries 
may have several causes. Thus, forcing assignment of a 
perinatal death into a single underlying cause, as 
required for ICD-10, may be less useful than previously 
appreciated [14,28]. For example, a combination of pre- 
maturity, birth asphyxia, and infection may coexist in an 
END, and it may be more useful from a public health 
policy perspective to consider all causes of death collec- 
tively. Additionally, some authors use final COD instead 
of underlying COD assignments [27]. 

Although the use of nonphysician coders to assign 
COD may not be a suitable alternative to the use of 
physician coders, other alternatives may have a role. A 
number of computer simulation techniques have been 
developed that address multiple COD and CSMF. Byass 
et al. have described a Bayesian approach called InterVA 
that simultaneously adjusts the probability of a finite list 
of causes according to affirmative answers to specific 



symptoms [29]. This approach calculates the likelihood 
of each COD and displays as many as three of the most 
probable COD, along with their associated likelihoods 
[11]. More recently, King and Lu developed an alterna- 
tive probabilistic method which directly estimates CSMF 
without individual COD attribution [1]. Data on symp- 
toms reported by caregivers along with COD are col- 
lected, and the COD distribution is estimated in the 
population in which only symptom data are available. 
Each of these methods has its advantages and drawbacks 
in terms of cost effectiveness, complexity, repeatability, 
and validity. For example, the King-Lu method depends 
on the availability of high-quality, facility-based, or valid 
mortality data, which are lacking in most settings where 
VA is needed. 

A major strength of this study is the use of a standar- 
dized VA training program for both nonphysicians and 
physicians. There are some limitations to this study. We 
did not use the harmonized VA questionnaire developed 
and published by WHO, since this was published after 
the start of the study. However, we believe that the 
results of our study would have been similar had we used 
this tool because the tools are broadly similar. Other lim- 
itations of this study include the lack of available medical 
diagnostic aids (laboratory, radiologic, or microbiologic 
studies) and lack of a postmortem examination for vali- 
dating the underlying COD assigned by physicians. 
Although the COD determined by the physician panel is 
often the traditional reference standard in VA methodol- 
ogy, physician panels have their limitations: their assign- 
ments of COD may contain systematic biases, they may 
not readily code diseases unexpected in certain demo- 
graphic groups, they tend to focus on the presence rather 
than the absence of symptoms, and they show a prefer- 
ence for highly specific diagnosis [12,30,31]. It is concei- 
vable that direct interactions between the community 
coordinator and the respondents (mothers and birth 
attendants) may have provided the community coordina- 
tors with more information from the respondents and 
the environment about the circumstances, signs, and 
symptoms of the deceased before death than was 
recorded on the standardized VA questionnaire. 

Verbal autopsy has been used in a variety of ways, 
including: to determine priority diseases and program- 
matic intervention; to conduct rapid assessments in 
emergency/ disaster situations; as sample registration of 
vital events; and perhaps most importantly, to describe 
population-level CSMF. If Millennium Development 
Goals related to pregnancy outcomes are to be achieved, 
it is imperative to understand more about the perinatal 
CODs which contribute disproportionately to under-5 
mortality in low-income countries. The shortage of 
human and material resources makes routine perinatal 
autopsies for deaths that occur in a community setting 
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in low-income countries unlikely. The relatively limited 
number of symptoms and signs exhibited by the fetus 
and neonate compared to adults and the strength of stu- 
dies conducted make perinatal verbal autopsy an attrac- 
tive medium-term alternative. 

Conclusions 

The most common method of assigning COD is the use 
of physician panels, which are costly and utilize scarce 
physician availability. In this study, the COD assigned 
by nonphysicians agreed with the COD determined by 
physicians about 50% of the time, and only tetanus and 
cord prolapse met robustness criteria. Although it may 
be too early to recommend against using nonphysicians 
to determine perinatal COD, based on our data, we 
recommend that further research be performed before 
nonphysicians are asked to determine perinatal COD in 
any settings in low-income countries. 

Funding 

Funding was provided by grants from the National Insti- 
tutes of Child Health and Human Development (U01 
HD 40636) and the Bill & Melinda Gates Foundation. 

Acknowledgements 

The authors would like to thank all the NICHD Global Network staff, 
community health workers, nurses, midwives, and physicians. Most of all, we 
thank the mothers and their babies. 

Author details 

'Departments of Pediatrics and Maternal Child Health, University of North 
Carolina at Chapel Hill, North Carolina, USA. 2 Kinshasa School of Public 
Health, Kinshasa, Democratic Republic of Congo, department of 
Community Health Sciences, The Aga Khan University, Karachi, Pakistan. 
4 IMSALUD/San Carlos University, Guatemala City, Guatemala, department of 
Pediatrics and Child Health, University Teaching Hospital, Lusaka, Zambia. 
6 Research Triangle Institute, Durham, North Carolina, USA. 'institute of 
Nutrition for Central America and Panama (INCAP), Guatemala City, 
Guatemala. 8 College of Medicine, Drexel University, Philadelphia, 
Pennsylvania, USA. 'Department of Pediatrics, University of Alabama at 
Birmingham, Alabama, USA. 10 Eunice Kennedy Shriver National Institute of 
Child Health and Human Development, Bethesda, Maryland, USA. 

Authors' contributions 

CE, VT, EM, DW, CB, WC, RG and LL had significant intellectual input in the 
conception and design of the study, data acguisition and analyses, draft 
writing, and final approval of the study. AG, U, JD, MP, MM, EC, OP and AT 
had significant input in the design of this study, data acquisition and 
analyses, draff writing, and final approval of the study. All authors read and 
approved the final manuscript. 

Competing interests 

The authors declare that they have no competing interests. 

Received: 24 March 201 1 Accepted: 5 August 201 1 
Published: 5 August 2011 

References 

1. Lopez AD, Mathers CD: Measuring the global burden of disease and 
epidemiological transitions: 2002-2030. Ann Trop Med Parasitol 2006, 
100:481-99. 



2. Engmann C, Matendo R, Kinoshita R, et al: Stillbirth and early neonatal 
mortality in rural Central Africa. Int J Gynaecol Obstet 2009, 1 05:1 1 2-7. 

3. Mathers CD, Fat DM, Inoue M, Rao C, Lopez AD: Counting the dead and 
what they died from: an assessment of the global status of cause of 
death data. Bull World Health Organ 2005, 83:171-7. 

4. Setel PW, Macfarlane SB, Szreter S, et al: A scandal of invisibility: making 
everyone count by counting everyone. Lancet 2007. 

5. Lawn J, Shibuya K, Stein C: No cry at birth: global estimates of 
intrapartum stillbirths and intrapartum-related neonatal deaths. Bull 
World Health Organ 2005, 83:409-17. 

6. Lawn JE, Osrin D, Adler A, Cousens S: Four million neonatal deaths: 
counting and attribution of cause of death. Paediatr Perinat Epidemiol 
2008, 22:410-6. 

7. Hill K, Lopez AD, Shibuya K, et al: Interim measures for meeting needs for 
health sector data: births, deaths, and causes of death. Lancet 2007, 
370:1726-35. 

8. Chandramohan D, Setel P, Quigley M: Effect of misclassification of causes 
of death in verbal autopsy: can it be adjusted? Int J Epidemiol 2001, 
30:509-14. 

9. Baiden F, Bawah A, Biai S, et al: Setting international standards for verbal 
autopsy. Bull World Health Organ 2007, 85:570-1. 

10. Garenne M, Fauveau V: Potential and limits of verbal autopsies. Bull World 
Health Organ 2006, 84:164. 

1 1 . Byass P, Fottrell E, Dao LH, et al: Refining a probabilistic model for 
interpreting verbal autopsy data. Scand J Public Health 2006, 34:26-31. 

12. Fantahun M, Fottrell E, Berhane Y, Wall S, Hogberg U, Byass P: Assessing a 
new approach to verbal autopsy interpretation in a rural Ethiopian 
community: the InterVA model. Bull World Health Organ 2006, 84:204-10. 

13. Freeman JV, Christian P, Khatry SK, et al: Evaluation of neonatal verbal 
autopsy using physician review versus algorithm-based cause-of-death 
assignment in rural Nepal. Paediatr Perinat Epidemiol 2005, 19:323-31. 

14. King G, Lu Y, Shibuya K: Designing verbal autopsy studies. Popul Health 
MetrlOW, 8:19. 

15. Quigley MA, Chandramohan D, Setel P, Binka F, Rodrigues LC: Validity of 
data-derived algorithms for ascertaining causes of adult death in two 
African sites using verbal autopsy. Prop Med Int Health 2000, 5:33-9. 

16. Chu K, Rosseel P, Gielis P, Ford N: Surgical task shifting in Sub-Saharan 
Africa. PLoS Med 2009, 6:el 000078. 

17. Labhardt ND, Balo JR, Ndam M, Grimm JJ, Manga E Task shifting to non- 
physician clinicians for integrated management of hypertension and 
diabetes in rural Cameroon: a programme assessment at two years. BMC 
Health Sen Res 2010, 10:339. 

18. Enweronu-Laryea C, Engmann C, Osafo A, Bose C: Evaluating the 
effectiveness of a strategy for teaching neonatal resuscitation in West 
Africa. Resuscitation 2009, 80:1308-11. 

19. Engmann C, Jehan I, Ditekemena J, et al: Using verbal autopsy to 
ascertain perinatal cause of death: are trained nonphysicians adequate? 
Trop Med Int Health 2009, 14:1496-504. 

20. Carlo WA, Goudar SS, Jehan I, et al: Newborn-care training and perinatal 
mortality in developing countries. N Engl J Med 2010, 362:614-23. 

21. Soleman N, Chandramohan D, Shibuya K Verbal autopsy: current 
practices and challenges. Bull World Health Organ 2006, 84:239-45. 

22. Lee AC, Mullany LC, Tielsch JM, et al: Verbal autopsy methods to ascertain 
birth asphyxia deaths in a community-based setting in southern Nepal. 
Pediatrics 2008, 1 21 :el 372-80. 

23. Fottrell E, Byass P: Verbal autopsy: methods in transition. Epidemiol Rev 
2010, 32:38-55. 

24. Engmann C, Jehan I, Ditekemena J, et al: An alternative strategy for 
perinatal verbal autopsy coding: single versus multiple coders. Tropical 
Medicine and International Health 201 1, 16:18-29. 

25. Agresti A: Categorical Data Analysis. John Wiley & Sons Incorporated. (New 
York): 1990. 

26. Setel PW, Whiting DR, Hemed Y, et al: Validity of verbal autopsy 
procedures for determining cause of death in Tanzania. Trop Med Int 
Health 2006, 11:681-96. 

27. Chowdhury HR, Thompson SC, Ali M, Alam N, Yunus M, Streatfield PK: A 
comparison of physicians and medical assistants in interpreting verbal 
autopsy interviews for allocating cause of neonatal death in Matlab, 
Bangladesh: can medical assistants be considered an alternative to 
physicians? Popul Health Metr 2010, 8:23. 



Engmann et al. Population Health Metrics 201 1, 9:42 
http://www.pophealthmetrics.eom/content/9/1/42 



Page 10 of 10 



28. Joshi R, Kengne AP, Neal B: Methodological trends in studies based on 
verbal autopsies before and after published guidelines. Bull World Health 
Organ 2009, 87:678-82. 

29. Byass P, Kahn K, Fottrell E, Collinson MA, Tollman SM: Moving from data 
on deaths to public health policy in Agincourt, South Africa: approaches 
to analysing and understanding verbal autopsy findings. PLoS Med 2010, 
7:e1 000325. 

30. Kahn K, Tollman SM, Garenne M, Gear JS: Validation and application of 
verbal autopsies in a rural area of South Africa. Trap Med Int Health 2000, 
5:824-31. 

31. Snow RW, Basto de Azevedo I, Forster D, et al: Maternal recall of 
symptoms associated with childhood deaths in rural east Africa. Int J 

Epidemiol 1993, 22:677-83. 

doi: 1 0. 11 86/1 478-7954-9-42 

Cite this article as: Engmann et al: Classifying perinatal mortality using 
verbal autopsy: is there a role for nonphysicians? Population Health 
Metrics 201 1 9:42. 



Submit your next manuscript to BioMed Central 
and take full advantage of: 

• Convenient online submission 

• Thorough peer review 

• No space constraints or color figure charges 

• Immediate publication on acceptance 

• Inclusion in PubMed, CAS, Scopus and Google Scholar 

• Research which is freely available for redistribution 



Submit your manuscript at f\ RioM _-| rpntr ,i 

www.biomedcentral.com/submit \ J •*"»■»•«• central 



